A general minimax result for relative entropy

Suppose Nature picks a probability measure P on a complete separable metric space X at random from a measurable set P fP g Then without knowing a statistician picks a measure Q on X Finally the statistician su ers a loss D P jjQ the relative entropy between P and Q We show that the minimax and maximin values of this game are always equal and there is always a minimax strategy in the closure of the set of all Bayes strategies This generalizes previous results of Gallager and Davisson and Leon Garcia Index terms minimax theorem minimax redundancy minimax risk Bayes risk relative entropy Kullback Leibler divergence density estimation source coding channel capacity computational learning theory